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FOREWORD 


Advanced  technology  can  tremendously  improve  operator  pro¬ 
ductivity.  However,  if  used  improperly,  it  can  be  an  ineffective 
and  costly  investment.  To  have  its  greatest  benefit,  advanced 
technology  must  be  implemented  where  it  can  provide  the  greatest 
payoff. 

This  report  describes  a  study  of  the  Improved  Guardrail  V 
System  operator  function.  The  effort  was  undertaken  at  the  re¬ 
quest  of  the  Communications  and  Electronic  Command  (CECOM)  Center 
for  Electronic  Warfare/Reconnaissance,  Surveillance,  and  Target 
Acquisition  (EW/RISTA)  as  part  of  their  program  to  determine 
where  emerging  technologies  could  be  used  to  enhance  operator 
performance  in  the  Guardrail/ Common  Sensor  System  5,  the  final 
planned  version  of  the  Guardrail/Common  Sensor  to  be  fielded. 

The  results  of  this  study  were  provided  to  RISTA  for  inclusion 
in  their  GR/CS  System  5  studies. 
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OPERATOR  PERFORMANCE  ENHANCEMENT  FOR  THE  GUARDRAIL/ COMMON  SENSOR 
SYSTEM  5 


EXECUTIVE  SUMMARY 


Requirement: 

The  Communications  and  Electronic  Command  (CECOM)  Center  for 
Electronic  Warfare/Reconnaissance,  Surveillance,  and  Target  Ac¬ 
quisition  (EW/RISTA)  will  employ  new  technologies  in  the  devel¬ 
opment  of  the  Guardrail/Common  Sensor  (GR/CS)  System  5.  As  a 
result,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  Fort  Huachuca  Field  Unit,  was  asked  to  identify 
where  advanced  technology  was  needed  to  improve  operator 
performance. 


Procedure: 

Interviews  with  subject  matter  experts  and  reviews  of  per¬ 
tinent  documents  were  carried  out  to  determine  the  analytical 
functions  of  the  Guardrail  Ground  Station.  Operators  were  inter¬ 
viewed  and  observed  while  performing  their  duties.  Operator 
workload  in  the  current  improved  Guardrail  system  was  assessed  to 
determine  the  extent  of  the  job  demands  placed  on  operators. 

Based  on  that  data,  it  was  determined  where  advanced  technology 
could  be  used  to  enhance  operator  performance. 


Findings: 

Results  indicated  that  workload  demands  were  not  excessively 
high  for  the  operator  activities,  and  that  while  full  workstation 
capability  was  not  used,  the  greatest  need  for  advanced  technol¬ 
ogy  is  for  those  functions  requiring  cognitive  skills  such  as 
analysis  and  foreign  language  interpretation. 


Utilization  of  Findings: 

The  recommendations  of  this  report  are  to  be  incorporated 
into  a  RISTA  report  on  hardware/software  technology  recommenda 
tions  for  the  GR/CS  System  5. 
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OPERATOR  PERFORMANCE  ENHANCEMENT  FOR  THE 
GUARDRAIL/ COMMON  SENSOR  SYSTEM  5 


Introduction 

The  Guardrail/Common  Sensor  (GR/CS)  will  be  the  next 
generation  Army  Airborne  SIGINT  collection  and  direction-finding 
system  replacing  Guardrail,  Improved  GR  V,  and  Quicklook  II.  The 
Guardrail  family  of  sensors  was  first  fielded  in  the  1970's  using 
technologies  of  the  late  1960  vintage.  During  the  evolution  of 
the  system,  new  processing  capabilities  were  added  and  the  number 
of  computers  and  their  technology  were  upgraded.  Currently, 

CECOM  Center  for  Electronic  Warfare/Reconnaissance,  Surveillance, 
and  Target  Acquisition  (EW/RISTA)  is  (1)  surveying  existing 
tactical  SIGINT  systems  to  identify  technical  approaches, 
equipment,  or  system  capabilities  which  could  be  used  in  the 
GR/CS  and  (2)  identifying  new  emerging  technologies  and  methods 
from  SIGINT  research  and  development  activities.  The  objective 
is  to  determine  what  and  how  advanced  technical  capabilities  can 
be  employed  in  the  development  of  the  GR/CS  System  5,  the  final 
planned  version  of  GR/CS  to  be  fielded. 

While  the  Army  is  pushing  to  increase  system  productivity 
through  the  introduction  of  new  materiel  developments,  it  is 
simultaneously  reducing  manpower.  As  a  result,  ARI  was  asked  by 
the  US  Army  Intelligence  Center  and  School  to  determine  if  the 
current  four  MOS  (Military  Occupation  Specialties)  assigned  to 
Guardrail  could  be  combined  into  a  single  generic  Common  Sensor 
Operator  MOS  for  the  future  GR/CS.  An  analysis  of  the  baseline 
Guardrail  or  Improved  GR  system  was  conducted  by  Hall  and  Mather 
(1987).  They  found  that  with  the  introduction  of  additional 
technology,  the  correct  allocation  of  soldier/machine  functions, 
and  task  automation,  the  number  of  MOS  required  to  operate  the 
GR/CS  could  be  reduced.  These  findings  indicate  that  while 
technology  was  used  to  upgrade  the  system  hardware,  there  was  not 
a  corresponding  advancement  in  the  software  and  soldier-machine 
interfaces. 


Objective 

As  a  result  of  the  1987  ARI  Study,  CECOM  EW/RISTA  requested 
ARI  to  identify  the  operator  requirements  for  the  Guardrail 
mission  and  determine  where  technology  could  be  used  to  enhance 
operator  performance.  The  results  of  this  study  are  to  be 
contrasted  with  the  hardware/software  technology  recommendations 
for  GR/CS  System  5. 
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Methodology 


The  GR/CS  system  is  composed  of  two  major  platforms,  the 
aircraft  which  houses  the  sensors  and  the  Integrated  Ground 
Processing  Facility  (IPF)  where  operators  analyze  the  sensor  data 
collected.  To  determine  where  technology  could  enhance  operator 
performance,  the  following  steps  were  performed: 

1.  Identify  IPF  operator  functions  during  GR/CS  mission. 

2.  Define  the  primary  activities  associated  with  each 
function. 

3.  Collect  operator  data  via  interviews  and  observations  of 
IGRV  operations. 

4.  Conduct  operator  workload  analysis. 

5.  Derive  where  operator  performance  could  be  enhanced  by 
new  technology. 

The  results  of  each  of  these  research  steps  are  described  in 
detail  below. 


Identify  IPF  Analytical  Functions 

For  this  step,  an  analytical  function  was  defined  as 
collection  of  activities  or  processes  that  occur  over  time 
without  implying  how  they  will  be  accomplished  or  what  instrument 
or  methods  will  be  used  to  perform  them.  The  IPF  functions  were 
derived  using  the  IGRV  Operator  Manual,  Program  of  Instruction 
(POI) ,  interviews  with  SMEs  at  the  US  Army  Intelligence  Center 
and  School,  and  the  US  Army  Intelligence  School-Ft.  Devens,  and  a 
job  analysis  of  IGRV  done  by  ARI.  From  these  data  it  was 
determined  that  the  IGRV  system  requires  seven  unique  analytical 
functions  to  be  performed.  These  functions  (shown  in  Figure  1) 
are  accomplished  in  a  sequential  but  iterative  manner  throughout 
the  mission.  A  brief  description  of  each  function  is  provided  in 
Table  1.  Although  the  IPF  has  undergone  major  hardware/software 
changes  since  its  initial  development,  these  functions  have  not 
changed;  thus,  they  are  considered  to  be  the  generic  analytical 
functions  for  the  Guardrail  mission. 


Major  Activities  within  the  IPF 

To  understand  the  operator  requirements  for  the  Guardrail 
mission  each  of  the  seven  analytical  functions  were  categorized 
into  the  specific  activities  performed  by  the  operators  during  a 
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Table  1 


Description  of  Analytical  Functions  for  Guardrail  Ground  Station 


1.  PREPARE  SYSTEM  -  Set  up  equipment  for  collecting,  analyzing, 
and  disseminating  SIGINT  information. 

2.  ESTABLISH  PRIORITIES  -  Organize  the  information  that  needs  to 
be  collected  and  order  them  by  level  of  importance. 

3.  LOCATE  SIGNAL  -  Find  a  radio  frequency  of  interest  which  has 
voice  transmission  and  data  transmission  within  range  of  the 
system. 

4.  PROCESS  DATA  -  Copy  voice  transmissions  and  obtain  location 
of  transmission  site. 

5.  PREPARE  GIST  -  Translate  and  record  voice  transmission. 

6.  ANALYZE  GIST  -  Identify  unit  sending  voice  transmission. 

7.  CREATE  REPORT  -  Transmit  information. 


mission.  It  was  determined  that  the  objectives  of  this  project 
could  best  be  met  by  looking  at  functional  activities  rather  than 
system-specific  tasks  as  traditionally  done.  Factors  that  made  a 
task  analysis  inappropriate  were:  (1)  the  Standard  Operating 
Procedures  (SOP)  between  units  differed  considerably,  and  (2) 
future  as  well  as  currently  proposed  design  changes  alter 
operator  procedures  thus  making  task-based  recommendations  moot. 

To  identify  the  primary  activities  operators  performed  for 
each  of  the  seven  functions,  IGRV  operations  were  observed  at  1st 
MI  Battalion  (AE) ,  Wiesbaden,  Germany,  V  Corps,  and  2d  MI 
Battalion  (AE) ,  Stuttgart,  Germany,  VII  Corps.  Primary 
activities  were  defined  as  the  activities  necessary  to  accomplish 
an  analytical  function  without  reference  to  the  tasks  involved. 
The  eleven  identified  primary  activities  performed  by  operators 
are  described  in  Table  2.  As  can  be  seen,  some  functions  are 
represented  by  one  major  activity  and  others  by  several. 

Although  different  MOS  are  assigned,  according  to  TOE  (Table  of 
Organization  and  Equipment) ,  to  perform  specific  functional 
activities,  units  in  the  field  do  not  adhere  to  the  MOS 
assignments.  As  a  result,  this  study  viewed  functions  and 
activities  as  generic  and  as  being  performed  by  a  generic 
operator. 


Collect  Operator  Data:  Observations  and  Operator  Critiques 

To  determine  what  requirements  the  system  imposed  on  the 
operator,  IGRV  operations  at  V  and  VII  Corps  were  observed  and 
the  IGRV  operators  were  interviewed.  Since  performance  is 
determined  by  job  requirements,  tools  or  equipment  to  perform  the 
job,  and  an  operator's  ability  to  perform  the  job,  these  topic 
areas  were  used  to  organize  and  discuss  the  observational  and 
interview  data  collected. 

Job  Requirements 

Although  this  study  focuses  on  generic  functions,  it  is 
necessary  to  discuss  this  section  within  the  context  of  specific 
MOS  functions  because  that  is  how  job  requirements  are 
structured.  The  system  is  currently  configured  where  IPF  tasks 
are  allocated  between  four  MOS,  each  handling  specific  unique 
functions.  However,  as  mentioned  above,  the  system  is  not 
utilized  as  designed.  One  unit  did  not  use  05Ds  (EW/SIGINT 
Emitter  Locator/Identifier) ,  another  used  05Ds  to  collect  data 
for  national  agencies  instead  of  supporting  the  98G.  One  unit 
had  language  trained  98Cs  (EW/SIGINT  Analysts)  and  reconfigured 
the  tasks  so  that  the  98C  worked  directly  with  the  98G  (Voice 
Interceptor) .  In  this  case,  the  analyst  "pre-screened"  the  voice 
communications  and  directed  the  voice  intercept  operator  to  the 
radio  frequencies  of  interest. 
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Table  2 


Description  of  Major  Activities  Performed  by  the  Generic  Operator 


1.  GET  PRIORITIES  The  operator  gets  a  list  of 

targets  to  collect  on  or  a  list 
of  prioritized  targets.  From  the 
list,  the  operator  prioritizes 
the  data  he  collects. 

2.  SEARCH  &  LOCATE  SIGNAL  There  are  two  search  modes, 

directed  and  general. 

General  Search:  An  operator 
sequentially  steps  through 
the  range  of  frequencies 
until  a  voice  communication 
or  data  transmission  is 
detected.  If  the 
transmission  is  not  of 
interest  the  process  is 
continued. 

Directed  Search;  An 
operator  spins  the  dial  to 
specified  frequencies  until 
a  voice  communication  or 
data  transmission  is 
detected. 

3.  LISTEN  &  HANDCOPY  When  a  voice  communication  is 

detected  an  operator  with  the 
required  language  skills  copies 
the  transmitted  message. 

4.  TAKE  LOB  &  GET  FIX  For  a  voice  communication  of 

interest,  an  operator  pushes  a 
key  to  obtain  Lines-of-Bearing 
(LOB)  from  the  aircraft  sensors. 
The  key  is  continually  pushed 
until  enough  data  has  been 
acquired  for  the  system  to 
automatically  calculate  a  fix  on 
the  location  of  the  communication 
source. 

5.  TRANSLATE  &  GIST  The  handcopy  message  of  voice 

traffic  is  translated.  An 
abstract  of  the  translated 
message  is  created  on  the  system 
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word  processor 


6.  SEND  GIST 


7 .  SCREEN  GIST 


8 .  ID  UNITS 


9.  CREATE  KLEIGHT  &  TACREP 


10,11.  PREPARE  REPORTS 


The  abstract  is  forwarded  to  the 
analysts  using  the  system's 
electronic  mail.  The  command  to 
forward  the  mail  is  simply  a  key 
stroke . 

The  gist  is  reviewed  and  a 
judgment  is  made  as  to  whether  it 
should  be  forwarded  for  further 
analysis  and  reporting.  The 
judgments  are  based  upon  mission 
priorities  and  quality  of 
information. 

The  analyst  uses  the  information 
in  the  gist  to  make  an  order  of 
battle  match  to  the  information. 
He  tries  to  explain  as  much  about 
the  unit  as  he  can  in  terms  of 
the  who,  what,  when,  where,  and 
why. 

Using  a  standardized  format, 
message  is  created  reporting  on 
the  intercepted  communication. 

Using  standardized  formats,  end 
of  mission  reports  are  generated 
which  cov*  r  all  the  activities 
which  occurred  during  an  eight 
hour  shift. 
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When  queries  were  made  as  to  why  the  05D  was  not  used,  it  was 
stated  that  the  current  DF  algorithms  made  it  unnecessary  to 
employ  the  05Ds.  It  was  also  stated  that  the  98Cs  and  98Gs  could 
do  their  job  effectively  without  the  05D.  The  unit  which 
reconfigured  the  tasking,  where  98C  and  98Gs  worked  in  a 
partnership  relationship,  believed  that  the  new  arrangement  made 
them  more  effective  since  it  reduced  unnecessary  collection  of 
redundant  information  and  provided  a  better  focus  for  the 
decision  effort.  Whether  these  assumptions  and  changes  optimize 
system  performance  or  not  is  unknown.  Performance  comparisons  of 
the  different  MOS  configurations  would  need  to  be  assessed  in  a 
controlled  environment  with  quantitative  measures  and  criteria. 

Because  Guardrail  products  are  the  results  of  a  team  effort, 
the  exchange  of  information  among  team  members  was  examined. 
Specific  functions  which  required  "handing-off"  information 
included  "get  priorities"  where  the  98C  gives  the  98G  the 
frequency  priority  list,  "send  gist"  where  the  98G  forwards  his 
translation  to  the  98C,  and  when  an  05D  is  utilized  in  the 
system,  edited  fixes  are  put  into  the  98G  database.  While  not 
observed,  the  05D  hand-off  does  not  utilize  unnecessary  human 
resources.  Files  are  pulled  up,  edited,  and  replaced  in  the  98G 
database.  The  hand-off  of  priorities  was  not  critical  since  it 
occurred  at  the  beginning  of  the  shift.  Dialogue  occurred  in 
explaining  what  was  to  be  done  during  the  shift  (establish 
priorities) ,  and  was  deemed  necessary  to  convey  mission 
objectives  to  a  team. 

The  procedure  for  the  hand-offs  of  gist  information  was 
straight  forward  (i.e.,  push  a  key  to  forward  the  abstract  to  the 
analyst) .  However,  the  time  for  operators  to  prepare  the 
information  for  sending,  as  well  as  the  time  for  the  receiving 
operator  to  review  the  information,  must  be  considered.  For 
example,  if  50  gists  were  completed  in  an  hour  and  it  took  2-3 
minutes  to  record  the  translated  voice  communications,  1  1/2  to  2 
hours  per  shift  is  used  preparing  information  for  another  to 
review.  If  one  minute  is  needed  to  review  that  information  and 
to  determine  the  merit  as  to  whether  to  analyze  that  data, 
another  hour  is  used.  Thus,  a  total  of  2  1/2  to  3  hours  of  an 
operator's  time  in  an  8  hour  shift  could  be  dedicated  to  the 
hand-off  process  which  under  a  heavy  threat  load  might  be 
undesirable  and  inefficient.  This  could  be  the  reason  one  unit 
tried  to  restructure  the  task  assignments  between  the  98C  and 
98G.  It  should  be  noted  that  the  Air  Force  does  not  separate  its 
personnel  in  the  IGRV  by  98C  and  98G  functions;  one  person  is 
assigned  to  perform  both. 

Tools  and  Equipment 

Within  the  IGRV  system,  there  are  workstations  for  each  MOS. 
In  general,  the  workstation  provide  200  software  functions  for 
operators  to  use.  Operators  indicated  that  they  could  perform  40 
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of  these  functions  and  routinely  use  10-12.  When  asked  why  they 
didn't  learn  the  other  functions,  operators  stated  that  they 
could  do  the  job  adequately  without  them.  However,  operators 
did  complain  about  the  parameters  of  the  commands  not  being 
readily  obvious  and  that  the  help  command  was  of  little  value. 

It  was  apparent  that  the  operators  did  not  understand  the 
architecture  of  the  database.  In  addition,  operators  indicated 
that  it  took  3-6  months  to  become  proficient  with  the 
workstation,  and  that  all  training  occurred  on-the-job.  These 
may  be  the  contributing  factors,  as  to  why  only  20%  of  the 
workstation  capability  is  being  utilized.  Quantitative  data  is 
required  to  determine  whether  the  workstation  capability  is  over¬ 
built,  the  software  is  improperly  designed,  or  if  better  training 
programs  are  required. 

Operators  found  obtaining  lines-of -bearing  (LOB)  to  get  a  fix 
frustrating.  Frequently,  when  the  key  was  pushed  to  acquire  a 
LOB,  they  could  not  obtain  one  due  to  the  position  of  the 
aircraft.  As  a  result  of  not  knowing  what  was  occurring, 
operators  tended  to  stick  a  pencil  into  the  keyboard  jamming  the 
key  until  LOB  information  was  obtained. 

Operator  Abilities 

Specific  skills  and  knowledge  bases  are  required  to  conduct 
the  Guardrail  mission.  They  are:  mastery  of  a  foreign  language 
(verbal  communication) ,  analytical  skills,  order  of  battle 
knowledge,  and  direction  finding.  All  personnel  observed  in  the 
IGRV  system  appeared  to  have  the  appropriate  skills.  However,  it 
was  noted  that  certain  procedures  in  performing  the  activities 
were  inefficient  due  to  the  requirement  to  find  information  in 
other  external  documents  or  sources.  For  example,  linguists 
frequently  needed  to  look  up  words  for  translation  and  analysts 
needed  to  consult  other  documents  to  identify  a  unit. 


Conduct  Workload  Assessment 

In  the  field,  successful  operator  performance  depends  on 
obtaining  and  analyzing  information  about  OPFOR  threat  elements. 
Without  ground  truth  or  an  external  criterion  it  is  impossible  to 
determine  how  well  the  operator  has  performed.  In  the  absence  of 
external  criterion,  workload  analysis  provides  a  method  for 
determining  factors  which  can  effect  operator  performance. 
Typically,  high  workload  levels  result  in  poor  performance.  The 
level  of  operator  performance,  in  general,  is  determined  by  such 
factors  as  (1)  the  perceived  speed  and  accuracy  with  which  the 
activity  must  be  completed;  (2)  the  kind  of  effort  required,  and 
(3)  the  required  skills,  procedures,  and  knowledge. 

The  NASA-Task  Load  Index  (TLX)  was  selected  for  use  since  it 
could  be  easily  administered  in  the  field,  has  moderate  to  high 
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diagnosticity  for  pinpointing  the  factors  involved  in  high 
workload  and  can  discriminate  between  different  levels  of 
workload  (Lysaght,  et  al.,  1988).  In  a  recent  study  by  ARI,  TLX 
has  been  shown  to  be  a  more  valid  subjective  workload  technique 
than  others  (Hill,  et  al.  1988).  The  NASA-TLX  uses  bipolar 
scales  to  assess  task  workload  in  six  dimensions:  mental  demand, 
physical  demand,  temporal  demand,  performance,  effort,  and 
frustration.  The  TLX  is  administered  by  having  operators  rate 
each  task  performed  on  the  six  workload  dimensions  using  a  scale 
from  low  to  high.  The  operator  also  rates  the  workload 
dimensions  to  determine  the  weight  of  each  dimension  relative  to 
the  other  dimensions.  The  weightings  and  rating  for  each 
dimension  are  combined  to  produce  the  weighted  workload  rating 
for  the  task.  The  workload  can  vary  between  0  and  100,  the  zero 
indicating  low  workload,  the  100  high  workload.  Intermediate 
points  on  the  scale  are  interpreted  accordingly,  i.e.,  50  would 
be  a  medium  workload,  25  relatively  low,  75  relatively  high. 

The  combined  aggregate  response  of  multiple  operators  are 
compiled  and  weighted  to  provide  a  workload  index.  A  sample 
rating  sheet  can  be  found  in  the  Appendix.  The  definitions  of 
the  workload  factors  are  provided  in  Table  3. 

To  assess  workload  of  the  Guardrail  operators,  14  personnel 
(6-98Cs,  and  8-98Gs)  at  the  1st  and  2d  MI  Battalions,  rated  the 
functional  activities.  Figure  2  shows  the  workload  assignment 
for  each  of  the  activities.  As  can  be  seen  from  this  figure, 
when  used  with  Table  2,  the  areas  with  the  highest  workload 
requirements  were  "listen  and  handcopy,"  "translate  and  gist," 
"identify  unit,"  and  "create  STRUM."  The  first  two  tasks 
involved  using  language  skills,  the  third  required  analytical 
skills  and  order  of  battle  knowledge,  and  the  later  involves 
writing  a  report.  Overall,  none  of  the  tasks  were  perceived  as 
placing  excessive  or  very  low  workload  demands  on  the  operators. 

The  weights  of  the  factors  which  contributed  to  the  total 
workload  ratings  are  shown  in  Table  4.  High  scores  indicate  that 
all  factors  contributed  to  the  overall  workload  index  except  for 
the  performance  factor.  Low  scores  on  the  performance  factor 
indicated  operators  perceive  they  are  successful  in  accomplishing 
the  activities.  In  general,  the  higher  workload  was  primarily 
due  to  mental  demands,  temporal  demands  and  frustration.  Of  the 
four  highest  workload  tasks,  these  factors  accounted  for  most  of 
the  workload  (Figure  3) .  However,  effort  was  a  high  weighted 
factor  for  the  two  language  related  activities  (listen  and 
handcopy,  translate  and  gist) .  Three  additional  observations 
should  be  noted.  First,  operators  perceived  they  were  successful 
in  accomplishing  the  goals  of  the  activities.  Second,  create  End 
of  Mission  Report,  while  requiring  a  heavy  mental  and  temporal 
demand,  had  no  frustration  factor,  and  thus  reduced  the  total 
workload  index.  Lastly,  there  was  a  high  frustration  weighting 
for  the  activity  "take  LOBs  and  get  fix." 
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Table  3 


NASA-TLX  Rating  Scale  Definitions 


TITLE 

ENDPOINTS 

DESCRIPTIONS 

MENTAL  DEMAND 

Low/High 

How  such  mental  and  perceptual 
activity  was  required  (e.g., 
thinking,  deciding;,  calculating 
remembering,  looking,  searching, 
etc.)?  Was  the  task  easy  or 
demanding,  simple,  or  complex, 
exacting  or  forgiving? 

PHYSICAL  DEMAND 

Low/High 

How  much  physical  activity  was 
required  (e.g.,  pushing, 
pulling,  turning,  controlling, 
activating,  etc.)?  Was  the  task 
easy  or  demanding,  slow  or  brisk 
slack  or  strenuous,  restful  or 
laborious? 

TEMPORAL  DEMAND 

Low/High 

How  much  time  pressure  did  you 
feel  due  to  the  rate  or  pace  at 
which  the  tasks  or  task  elements 
occurred?  Was  the  pace  slow  and 
leisurely  or  rapid  and  frantic? 

PERFORMANCE 

Low/High 

How  successful  do  you  think  you 
were  in  accomplishing  the  goals 
of  the  task  set  by  the 
experimenter  (or  yourself)?  How 
satisfied  were  you  with  your 
performance  in  accomplishing 
these  goals? 

EFFORT 

Low/High 

How  hard  did  you  have  to  work 
(mentally  and  physically)  to 
accomplish  your  level  of 
performance? 

FRUSTRATION 

DEMAND 

Low/High 

How  insecure,  discouraged 
initiated,  stressed  and  annoyed 
versus  secure,  gratified, 
content,  relaxed  and  complacent 
did  you  feel  during  the  task? 
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Table  4 


Total  workload  and  the  contribution  of  the  workload  factors  to  the 
total,  for  the  11  IGRV  activities. 


ACTIVITIES 

TOTAL 

WORKLOAD 

WORKLOAD  FACTORS 

MD 

PD 

TD 

n 

E 

H 

1.  Get  Priorities 

41.7 

6.1 

0 

9.1 

9.1 

8.9 

8.5 

2.  Search  & 

Locate  Signals 

42.8 

9.0 

6.8 

9.9 

8.1 

3.  Listen  &  Hand-copy 

62.4 

14.6 

0 

16.3 

5.4 

n 

12.1 

4.  Take  LOBs  & 

Get  Fix 

45.7 

6.9 

0 

10.1 

6.8 

B 

12.5 

5.  Translate  &  Gist 

58.7 

14.5 

0 

15.9 

4.3 

11.7 

12.3 

6.  Send  Gist 

32.4 

B 

11.1 

H 

8.0 

B 

7.  Screen  Gist 

40.0 

10.0 

.2 

H 

9.9 

7.9 

6.3 

8.  ID  Units 

65.2 

17.0 

0 

19.3 

H 

8.1 

16.6 

9.  Create  KL  & 

TACREP 

33.5 

8.6 

0 

9.6 

B 

5.8 

5.3 

10.  Create  End  of 

Mission  Report 

45.1 

12.5 

0 

17.7 

B 

6.5 

4.1 

11.  Create  STRUM 

60.6 

16.9 

0 

17.0 

B 

8.7 

15.3 

MD  =  mental  demand  PD  =  physical  demand  TD  =  time  demand 
P  =  performance  E  =  effort  F  =  frustration 
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Analysis:  Technology  Implications 

Gilbert's  behavior  engineering  model  (Gilbert,  1978) 
indicates  that  there  are  two  ways  to  enhance  performance.  One  is 
by  providing  environmental  supports  such  as  supervision,  tools 
and  materials,  and  motivation.  The  other  is  by  manipulating  the 
required  behavior  through  training,  task  structure,  or  selection. 
When  these  behavioral  techniques  are  matched  to  the  workload 
dimensions,  guidance  is  provided  as  to  which  remedies  would  work 
better  for  specific  workload  demands  (see  Table  5) .  This  shows, 
for  example,  if  high  workload  ratings  are  due  to  mental  demands, 
job  aids  and  training  techniques  should  be  examined  for  the 
source  of  remedy.  For  activities  with  high  physical  demands,  job 
aids  and  selection  techniques  could  improve  performance. 

Job  aids  can  range  from  new  types  of  databases  to  computer 
software  and  automation,  including  revised  interface  designs, 
help  features,  and  tools  which  can  be  provided  to  operators. 
Supervision  involves  letting  the  operator  know  what  is  expected 
and  guides  him  to  meet  those  expectations.  Motivation  refers  to 
appropriate  reinforcement  for  performance.  Training,  either 
formal  or  informal,  is  used  to  increase  skill  level.  Task 
structure  involves  the  design  and  sequencing  of  procedures  that 
the  operator  must  perform  in  order  to  complete  the  task. 

Finally,  selection  refers  to  matching  the  operator  to  the  job 
both  physically  and  psychologically. 

Although  there  are  many  behavioral  engineering  techniques 
which  can  enhance  operator  performance,  only  supporting  the 
operator  with  job  aids  meets  the  technological  enhancement 
criteria.  Since  the  ratings  of  all  the  activities  fell  in  the 
mid-workload  range,  and  since  job  aids  are  the  primary  remedies 
for  most  workload  factors,  each  activity  will  be  discussed 
individually. 

Get  Priorities.  Ratings  on  each  of  the  factors  were  on  the 
low  end  of  the  scale.  Although  operators  indicated  that  they  did 
not  perform  the  task  as  well  as  others,  performance  was  still 
high.  Thus,  the  introduction  of  technology  is  unnecessary  for 
this  activity. 

Search  and  Locate  Signal.  Again  the  ratings  indicated  no 
problem;  however,  this  activity  was  ranked  third  in  effort. 

Since  job  aids,  as  shown  in  Table  5,  can  be  used  to  reduce  effort 
workload,  technological  implications  were  explored.  One  major 
observation  was  noted.  During  general  search  mode,  the 
sequential  stepping  through  frequencies  was  detrimental.  A 
potential  solution  would  be  to  use  a  scanner  which  would 
automatically  search  for  a  frequency  with  a  signal.  Directed 
search  did  not  have  the  same  problems  but  a  scanner  could  still 
be  utilized  if  programmable. 
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Table  5 


Behavioral  Engineering  Techniques  Which  Best  Address  the  NASA 
Workload  Factors 


TECHNIQUES  FOR  BEHAVIORAL 

ENGINEERING  OF 

FACTOR 

v#*  ?  ■■  v'T*  •'rv?  W  f  xt 

Mental  Demand 

Job  Aids 

Training 

Physical  Demand 

Job  Aids 

Selection 

Temporal  Demand 

Job  Aids 

Task  Structure 

Performance 

Supervision, 

Motivation 

Training,  Selection 

Effort 

Job  Aids 

Task  Structure 

Frustration 

Supervision 

Task  Structure, 
Structure 
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Listen  and  Handcopv.  This  activity  and  "translate  and  gist" 
were  combined  because  they  involved  the  same  cognitive  skills. 
They  also  had  the  same  workload  demands ,  involved  with  language 
skills,  and  require  concentration.  Although  research  is  being 
done  to  develop  computerized  language  translators,  the  technology 
is  not  advanced  enough  to  be  considered  for  implementation.  Thus 
the  only  support  technology  which  could  currently  be  provided  the 
voice  interceptor  is  in  the  form  of  a  computerized  dictionary. 
Such  a  dictionary  would  allow  an  operator  to  search  out  words  and 
idioms  for  translation,  military  meanings,  etc.  Such  support 
would  reduce  the  time  now  taken  to  search  other  documents. 

Take  Lob  and  Get  Fix.  Only  the  frustration  and  effort 
factors  contributed  to  the  workload  of  this  activity.  The  reason 
is  due  to  the  futile  attempts  to  get  LOBs  as  mentioned  before. 
Some  other  mechanism  should  be  used,  perhaps  software  functions 
or  some  feedback  mechanism,  to  avoid  operators  jamming  the  key  in 
order  to  not  push  it  continually. 

Translate  and  Gist.  See  Listen  and  Handcopy. 

Send  Gist.  This  activity  had  the  lowest  workload  rating  of 
all  the  activities  with  time  demand  being  the  primary  contributor 
to  the  rating.  Since  time  constraints  are  inherent  in  the  nature 
of  the  job  and  the  workload  is  low,  no  new  technology  needs  to  be 
employed. 

Screen  Gist.  While  the  activity  had  a  low  workload,  it 
involves  the  hand-off  process  previously  discussed.  The  review 
process  can  be  made  easier  if  within  the  preparation  process  key 
elements  of  the  Gist  can  be  automatically  highlighted.  Then, 
these  key  areas  can  be  automatically  related  to  the  priorities, 
e.g.,  through  an  AI  program.  It  should  be  possible  to  have  the 
program  prioritize  Gists  for  analyses. 

ID  Units.  As  with  the  language  activities,  the  analysis 
activity  had  a  high  workload  index.  Technology  which  is  feasible 
in  the  near  future  would  be  an  automated  database  for  Order  of 
Battle  data.  With  search  capabilities  that  allow  for  obtaining 
information,  call  sign,  frequencies,  activities,  etc.,  the 
analytical  function  could  be  performed  more  efficiently.  In  the 
future,  other  technologies  such  as  artificial  intelligence  or 
decision  augmentation  systems  could  be  used  to  augment  the 
operator. 

Create  Reports.  The  last  three  activities,  create  KL  and 
TACREP,  create  End  of  Mission  Report,  and  create  STRUM  all 
involve  report  writing.  They  differ  in  how  much  information  and 
what  information  is  required  to  go  into  the  report.  It  should  be 
possible  to  activate  the  report  writing  to  the  extent  that  as 
information  is  recorded  and  processed  throughout  the  activity 
cycle,  it  could  be  automatically  collated  and  categorized  as  it 
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is  produced.  If  information  doesn’t  get  past  an  activity,  e.g., 
a  gist  is  screened  as  not  to  be  analyzed,  and  that  information  is 
not  needed  for  one  of  the  final  reports,  i.e.,  is  not  relevant  to 
any  of  the  priorities,  it  could  be  automatically  eliminated.  An 
automated  system  to  write  the  report  would  address  the  time 
demands  and  any  frustration  due  to  gathering  and  recollating 
information  that  had  already  been  processed.  In  turn,  the  mental 
demand  would  change  from  a  production  activity  (writing)  to  being 
an  editing  activity. 


Conclusions 

When  advanced  technologies  are  incorporated  in  existing  or 
new  Army  systems,  the  effectiveness  is  accomplishing  mission 
objectives  and  the  efficiency  in  achieving  those  objectives 
should  increase.  For  this  study  on  Guardrail/Common  Sensor  5,  it 
was  impossible  to  assess  the  effectiveness  of  implementing 
advanced  technologies.  However,  inefficiencies  were  identified 
in  terms  of  time,  personnel,  and  materiel  usages  on  the  current 
Guardrail  system.  These  inefficiencies  could  be  improved  to  a 
limited  degree  by  the  introduction  of  new  technologies.  However, 
the  data  suggests  that  the  current  TOE  may  not  be  optimal  and  the 
workstation  is  either  over-designed  or  personnel  are  not 
adequately  trained.  Changes  made  in  the  equipment  which 
operators  use  must  be  done  in  parallel  with  a  review  of  personnel 
and  training  requirements.  This  includes  not  only  the  operator 
functions  examined  in  this  study  but  also  the  maintainer 
functions.  Problems  already  exist  in  maintaining  the  IGRV.  As 
equipment  becomes  more  sophisticated  in  assisting  the  operator, 
greater  demands  may  be  placed  on  the  maintainers,  the  system  for 
training  the  maintainers  and  operators,  and  the  resource 
requirements  necessary  to  field  an  effective  work  force  increase 
accordingly.  To  determine  the  value  of  introducing  new 
technology,  trade-off  analyses  of  time  saved,  personnel  resources 
used,  and  materiel  cost  must  be  made. 
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APPENDIX  A 


NASA  TLX  RATING  SHEET 


MOS_ 

TASK 


RATING  SHEET 

MENTAL  DEMAND 

I  1  I  1  I  I  I  1  I  1  I  1  l  I  I  I  I  1  J.  - 

LOW  HIGH 

PHYSICAL  DEMAND 

i  I  i  I  i  1  i  I  i  1  i  I  i  I  i  I  i  I  i 

LOW  HIGH 

TEMPORAL  DEMAND 

i  I  i  I  i  I  i  I  i  I  i  I  i  I  i  I  i  I  i 

LOW  HIGH 

PERFORMANCE 

i  I  i  I  i  I  i  1  i  I  i  1  i  I  i  I  i  I  i 

good  poor 

EFFORT 

i  1  i  1  i  I  i  I  i  i  I  i  1  i  I  i  I  i 

LOW  HIGH 

FRUSTRATION 

i  I  i  I  i  I  i  I  i  1  i  I  i  I  i  I  i  I  i 

LOW  HIGH 
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